根据认知心理学和相关学科，生物学剂中复杂的解决问题行为的发展取决于等级认知机制。分层增强学习是一种有前途的计算方法，最终可能在人工代理和机器人中产生可比的解决问题的行为。但是，迄今为止，许多人类和非人类动物的解决问题能力显然优于人造系统的能力。在这里，我们提出了整合生物学启发的层次机制的步骤，以实现人造代理中的高级解决问题的技能。因此，我们首先回顾了认知心理学中的文献，以强调构图抽象和预测性处理的重要性。然后，我们将获得的见解与当代分层的强化学习方法联系起来。有趣的是，我们的结果表明，所有确定的认知机制均已在孤立的计算体系结构中单独实施，这提出了一个问题，为什么没有单个统一体系结构可以集成它们。作为我们的最终贡献，我们通过对开发这种统一体系结构的计算挑战的综合观点来解决这个问题。我们希望我们的结果可以指导更复杂的认知启发的分层机器学习体系结构的发展。

人类可以在各种时间尺度和层次级别上做出预测。因此，对事件编码的学习似乎起着至关重要的作用。在这项工作中，我们通过自主学习的潜在事件代码对层次预测的开发进行建模。我们提出了分层复发性神经网络结构，其诱导学习偏见促进了压缩感觉运动序列的稀疏潜在状态的发展。更高级别的网络学会了预测潜在国家倾向于改变的情况。使用模拟机器人操纵器，我们证明系统（i）学习了准确反映数据事件结构的潜在状态，（ii）在较高级别上开发有意义的时间抽象预测，（iii）生成了靶心，相似的行为在与婴儿的眼神追踪研究中发现的凝视行为。该体系结构为自主学习收集的经验的压缩层次编码以及对这些编码产生适应性行为的开发提供了一步。

灵活的目标指导行为是人类生活的一个基本方面。基于自由能最小化原理，主动推断理论从计算神经科学的角度正式产生了这种行为。基于该理论，我们介绍了一个输出型，时间预测的，模块化的人工神经网络体系结构，该建筑处理感觉运动信息，渗透到世界上与行为相关的方面，并引起高度灵活的，目标定向的行为。我们表明，我们的建筑经过端对端训练，以最大程度地减少自由能的近似值，它会发展出可以将其解释为负担能力地图的潜在状态。也就是说，新兴的潜在状态表明哪种行动导致哪些效果取决于局部环境。结合主动推断，我们表明可以调用灵活的目标指导行为，并结合新兴的负担能力图。结果，我们的模拟代理会在连续的空间中灵活地转向，避免与障碍物发生碰撞，并且更喜欢高确定性地导致目标的途径。此外，我们表明，学识渊博的代理非常适合跨环境的零拍概括：在训练少数固定环境中的代理商在具有障碍和其他影响其行为的固定环境中，它在程序生成的环境中表现出色，其中包含不同量的环境不同位置的各种尺寸的障碍和地形。

在部分可观察域中的预测和规划的常见方法是使用经常性的神经网络（RNN），其理想地开发和维持关于隐藏，任务相关因素的潜伏。我们假设物理世界中的许多这些隐藏因素随着时间的推移是恒定的，而只是稀疏变化。为研究这一假设，我们提出了Gated $ L_0 $正规化的动态（Gatel0rd），一种新的经常性架构，它包含归纳偏差，以保持稳定，疏口改变潜伏状态。通过新颖的内部门控功能和潜在状态变化的$ l_0 $ norm的惩罚来实现偏差。我们证明Gatel0rd可以在各种部分可观察到的预测和控制任务中与最先进的RNN竞争或优于最先进的RNN。 Gatel0rd倾向于编码环境的基础生成因子，忽略了虚假的时间依赖性，并概括了更好的，提高了基于模型的规划和加强学习任务中的采样效率和整体性能。此外，我们表明可以容易地解释开发的潜在状态，这是朝着RNN中更好地解释的步骤。

认知心理学和相关学科已经确定了几种关键机制，使智能生物学药物能够学会解决复杂的问题。存在紧迫的证据表明，这些物种中能够解决问题技能的认知机制以等级心理表征为基础。在为人工代理和机器人提供基于学习的问题解决能力的最有希望的计算方法之一是分层增强学习。但是，到目前为止，现有的计算方法尚未能够为人工代理提供与智能动物相媲美的解决问题的能力，包括人类和非人类灵长类动物，乌鸦或章鱼。在这里，我们首先调查了认知心理学和相关学科的文献，发现许多重要的心理机制涉及组成抽象，好奇心和前瞻性模型。然后，我们将这些见解与当代分层的增强学习方法联系起来，并确定实现这些机制的关键机器智能方法。作为我们的主要结果，我们表明所有重要的认知机制均已在孤立的计算体系结构中独立实施，并且缺乏适当整合它们的方法。我们希望我们的结果指导更复杂的认知启发性层次结构方法的发展，以便未来的人工代理在智能动物水平上实现解决问题的性能。

Robotic teleoperation is a key technology for a wide variety of applications. It allows sending robots instead of humans in remote, possibly dangerous locations while still using the human brain with its enormous knowledge and creativity, especially for solving unexpected problems. A main challenge in teleoperation consists of providing enough feedback to the human operator for situation awareness and thus create full immersion, as well as offering the operator suitable control interfaces to achieve efficient and robust task fulfillment. We present a bimanual telemanipulation system consisting of an anthropomorphic avatar robot and an operator station providing force and haptic feedback to the human operator. The avatar arms are controlled in Cartesian space with a direct mapping of the operator movements. The measured forces and torques on the avatar side are haptically displayed to the operator. We developed a predictive avatar model for limit avoidance which runs on the operator side, ensuring low latency. The system was successfully evaluated during the ANA Avatar XPRIZE competition semifinals. In addition, we performed in lab experiments and carried out a small user study with mostly untrained operators.

The purpose of this work was to tackle practical issues which arise when using a tendon-driven robotic manipulator with a long, passive, flexible proximal section in medical applications. A separable robot which overcomes difficulties in actuation and sterilization is introduced, in which the body containing the electronics is reusable and the remainder is disposable. A control input which resolves the redundancy in the kinematics and a physical interpretation of this redundancy are provided. The effect of a static change in the proximal section angle on bending angle error was explored under four testing conditions for a sinusoidal input. Bending angle error increased for increasing proximal section angle for all testing conditions with an average error reduction of 41.48% for retension, 4.28% for hysteresis, and 52.35% for re-tension + hysteresis compensation relative to the baseline case. Two major sources of error in tracking the bending angle were identified: time delay from hysteresis and DC offset from the proximal section angle. Examination of these error sources revealed that the simple hysteresis compensation was most effective for removing time delay and re-tension compensation for removing DC offset, which was the primary source of increasing error. The re-tension compensation was also tested for dynamic changes in the proximal section and reduced error in the final configuration of the tip by 89.14% relative to the baseline case.

Learning enabled autonomous systems provide increased capabilities compared to traditional systems. However, the complexity of and probabilistic nature in the underlying methods enabling such capabilities present challenges for current systems engineering processes for assurance, and test, evaluation, verification, and validation (TEVV). This paper provides a preliminary attempt to map recently developed technical approaches in the assurance and TEVV of learning enabled autonomous systems (LEAS) literature to a traditional systems engineering v-model. This mapping categorizes such techniques into three main approaches: development, acquisition, and sustainment. We review the latest techniques to develop safe, reliable, and resilient learning enabled autonomous systems, without recommending radical and impractical changes to existing systems engineering processes. By performing this mapping, we seek to assist acquisition professionals by (i) informing comprehensive test and evaluation planning, and (ii) objectively communicating risk to leaders.

In inverse reinforcement learning (IRL), a learning agent infers a reward function encoding the underlying task using demonstrations from experts. However, many existing IRL techniques make the often unrealistic assumption that the agent has access to full information about the environment. We remove this assumption by developing an algorithm for IRL in partially observable Markov decision processes (POMDPs). We address two limitations of existing IRL techniques. First, they require an excessive amount of data due to the information asymmetry between the expert and the learner. Second, most of these IRL techniques require solving the computationally intractable forward problem -- computing an optimal policy given a reward function -- in POMDPs. The developed algorithm reduces the information asymmetry while increasing the data efficiency by incorporating task specifications expressed in temporal logic into IRL. Such specifications may be interpreted as side information available to the learner a priori in addition to the demonstrations. Further, the algorithm avoids a common source of algorithmic complexity by building on causal entropy as the measure of the likelihood of the demonstrations as opposed to entropy. Nevertheless, the resulting problem is nonconvex due to the so-called forward problem. We solve the intrinsic nonconvexity of the forward problem in a scalable manner through a sequential linear programming scheme that guarantees to converge to a locally optimal policy. In a series of examples, including experiments in a high-fidelity Unity simulator, we demonstrate that even with a limited amount of data and POMDPs with tens of thousands of states, our algorithm learns reward functions and policies that satisfy the task while inducing similar behavior to the expert by leveraging the provided side information.

Speech-driven 3D facial animation has been widely explored, with applications in gaming, character animation, virtual reality, and telepresence systems. State-of-the-art methods deform the face topology of the target actor to sync the input audio without considering the identity-specific speaking style and facial idiosyncrasies of the target actor, thus, resulting in unrealistic and inaccurate lip movements. To address this, we present Imitator, a speech-driven facial expression synthesis method, which learns identity-specific details from a short input video and produces novel facial expressions matching the identity-specific speaking style and facial idiosyncrasies of the target actor. Specifically, we train a style-agnostic transformer on a large facial expression dataset which we use as a prior for audio-driven facial expressions. Based on this prior, we optimize for identity-specific speaking style based on a short reference video. To train the prior, we introduce a novel loss function based on detected bilabial consonants to ensure plausible lip closures and consequently improve the realism of the generated expressions. Through detailed experiments and a user study, we show that our approach produces temporally coherent facial expressions from input audio while preserving the speaking style of the target actors.